Method and apparatus for region-based weighted prediction with improved global brightness detection

ABSTRACT

A method and apparatus for determining a region-based weighted prediction with improved global brightness detection. The method includes applying a global brightness change detection methods by computing the weighted prediction parameters, determining if the brightness change is different amount of change for the different regions, if the change is not different, calculate motion estimation and setting Refidx to 1 and setting Refidx is set to 0 when there is change, determining the best motion vector, motion vector cost and the best reference input, and determining a region-based weighted prediction with improved global brightness detection based on the motion vector data.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims benefit of U.S. provisional patent applicationSer. No. 61/363,454, filed Jul. 12, 2010, which is herein incorporatedby reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

Embodiments of the present invention generally relate to a method andapparatus for region-based weighted prediction; more specifically, toregion-based weighted prediction with improved global brightnessdetection.

2. Description of the Related Art

Weighted prediction (WP) is introduced in H.264 video coding standard toimprove coding efficiency for the video sequences that have brightnesschanges between frames. It is found to provide high coding efficiencyfor the video sequence with global brightness changes. However, since WPin H.264 is slice based method, it also leads to significant coding lossfor the sequences that have lots of local brightness changes and highmotions when it is applied without caution.

Some embodiments of global brightness change detection (GBD) method toapply WP when there are global brightness change between the current andthe reference frames. Such embodiments are effective in not introducingcoding loss for normal sequences while improving coding efficiency forthe sequences with global brightness changes. However, such embodimentscould be worse than applying WP always especially when there are localbrightness changes.

Therefore, there is a need for a region-based weighted prediction withimproved global brightness detection.

SUMMARY OF THE INVENTION

Embodiments of the present invention relate to a method and apparatusfor determining a region-based weighted prediction with improved globalbrightness detection. The method includes applying a global brightnesschange detection methods by computing the weighted predictionparameters, determining if the brightness change is different amount ofchange for the different regions, if the change is not different,calculate motion estimation and setting Refidx to 1 and setting Refidxis set to 0 when there is change, determining the best motion vector,motion vector cost and the best reference input.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above recited features of the presentinvention can be understood in detail, a more particular description ofthe invention, briefly summarized above, may be had by reference toembodiments, some of which are illustrated in the appended drawings. Itis to be noted, however, that the appended drawings illustrate onlytypical embodiments of this invention and are therefore not to beconsidered limiting of its scope, for the invention may admit to otherequally effective embodiments.

FIG. 1 is an embodiment of a local brightness changes and association toreference index; and

FIG. 2 is an embodiment of a flow diagram depicting a method forregion-based weighted prediction with improved global brightnessdetection.

DETAILED DESCRIPTION

A region based WP method to handle local brightness change as well asimproved GBD method is proposed. First, improved GBD is employed todetermine if there is local brightness change and to identify suchregions. Once local brightness changes are found, region based WP isapplied using reference picture reordering to efficiently handle suchregions. Then, motion estimation (ME) for each block is performed overone reference index (frame) depending on whether the region, in whichthe block is located, has brightness change or not.

Such a solution is useful when there are strict limitations onresources, such as, cycle count, memory bandwidth, etc. In the currenthardware implementation of H.264 encoder, a single reference is used forME due to such resource constraints. Since one reference is loaded andone ME per block is needed, no additional computations and memorybandwidth for ME are needed.

H.264 supports reference picture reordering by which the referencepictures in decoded picture buffer (DPB) could be reordered if necessaryto improve coding efficiency. It is even possible that the samereference picture is placed in multiple reference indices. It is worthnoting that each reference picture may have its own WP parameters inH.264. Using these two H.264 features, the effective region may be basedon a WP method. One may assume that frame picture encoding with a singlereference frame and each frame consist of a single slice.

To improve global brightness change detection, one may divide thecurrent frame and the reference frame into the same arbitrary N regionsof any size and shapes. Then, apply the global brightness changedetection by which we can also identify the regions that have brightnesschanges can be identified by: (1) calculating the mean pixel values andvariances of the n-th region of current picture and reference picture,where they are denote by m_(cur)(n), m_(ref)(n), v_(cur)(n) andv_(ref)(n), respectively. Then, the absolute difference is calculatedbetween the two mean values, m_(diff)(n)=|m_(cur)(n)−m_(ref)(n)|; andthen (2) comparing the absolute mean difference, m_(diff)(n), to apredefined threshold, TH. If m_(diff)(n) is larger than TH, thenincrease the number of regions that have brightness change, Rcount,by 1. These steps may be repeated for a specific number or type ofregions or for all regions; and (3) checking if eitherv_(cur)(n)−v_(ref)(n)>=0 or v_(cur)(n)−v_(ref)(n)<=0 for all N regions.Based on the observation that, with fade-in and fade-out, the variancesof all regions either increase (fade-in) or decrease (fade-out), if themean pixel values changes are determined to be caused byfade-in/fade-out, then set Fade=1. Where if the Fade is set to 1 andRcount is larger than or equal to N_(max) (N_(min)<N_(max)<=N), wecompute WP parameters, associate them to refidx equal to 0 and encodeall MBs with motion estimation (ME) using refidx equal to 0. Otherwise,if Fade=0 or Rcount is less than or equal to N_(min), we turn off WP andencode all MBs with ME using refidx equal to 0.

In the region-based weighted prediction, if the fade is set to 1 andN_(min)<Rcount<N_(max), the reference frame is reordered in order forthe same reference frames to sit in two places, refidx=0 and refidx=1,respectively. In such a case, WP parameters are computed and associatedto refidx equal to 0 and zero WP parameters are associated to refldxequal to 1. The encoder may send reference reordering syntaxes fordecoders. However, when encoding each MB, if the MB is in the regionwith brightness change, ME is performed with refidx equal to 0. However,if the MB is in the regions without brightness change, ME is performedwith refidx equal to 1. In one embodiment, if 2 reference frames areused without global brightness change, coding gains are lost due to theincreased overhead bits to encode reference index. Such losses are verynoticeable with normal sequences that have no fade-in, fade-out andbrightness changes.

In one embodiment, one may consider the percentage of regions withbrightness change for it. For example, the number of regions withbrightness change is smaller than the other, zero WP may be associatedwith refidx=0. This is important because it affects the number ofskipped blocks, whose refidx should be 0.

FIG. 1 is an embodiment of a local brightness changes and association toreference index. In FIG. 1, the frames are divided into 5 rectangularregions; two of frames include luminance change (gray regions) and theothers do not (white regions). In such a case, the reference list isreordered to put the same reference frame (frame t−1) into refidx=1 aswell. Then, the computed WP parameters (WP0) is associated with refidx=0and zero WP parameters (WP1) with refidx=1. Then we perform ME usingrefidx=0 for MBs in the gray regions and refidx=1 for MBs in the whiteregions.

FIG. 2 is an embodiment of a flow diagram depicting a method 200 forregion-based weighted prediction with improved global brightnessdetection. The method starts at step 202 and continues to step 204. Atstep 204, the method 200 determines if a change in global brightness wasdetected. If a change was detected, the method 200 proceeds to step 206,wherein weighted prediction parameters are calculated. Otherwise, themethod 200 proceeds to step 208, wherein the method 200 determines localbrightness has changed. If there is no change, the method 200 proceedsto step 210, wherein the weighted prediction parameters are set to zero(0). Otherwise, the method 200 proceeds to step 222.

From steps 206 and 210, the method 200 proceeds to step 212, wherein themethod 200 sets micro block index to zero (0) and proceeds to step 214.At step 214, the method 200 sets motion estimation with refldx to zero(0). At step 216, the method 200 determines if it is the last microblock. If it is not the last block, the method 200 proceeds to step 220and sets micro block in increased by one (1). The method 200 proceedsfrom step 220 to step 214. From step 208, if the local brightnesschanged, the method 200 proceeds to step 222. At step 222, the method200 calculates the weighted prediction parameters and referencere-ordering. At step 224, the method 200 step the micro block index tozero (0). At step 226, the method 200 determines if the micro block isin the region of luminance change. It is in the region, the methodproceeds to step 230. At step 230, the method 200 calculates motionestimation with refidx set to zero (0). Otherwise, the method proceedsfrom step 226 to step 228, wherein the method 200 calculates motionestimation with refidx set to one (1) and weighted prediction parametersset to zero (0).

From steps 228 and 230, the method 200 proceeds to step 232, wherein themethod 200 determines if it is the last micro block. If it is not thelast micro block, the method 200 proceeds to step 234, wherein the microblock index is incremented by one (1). From step 234, the method 200returns to step 226. Otherwise, the method 200 proceeds from step 232 tostep 236. The method 200 ends at step 236.

To compute weighted prediction (WP) parameters, various method anddifferent regions may be considered for weighted prediction parametercalculation, such as, regions where brightness have changed or allregions regardless of how brightness changes locally. In one embodiment,N reference indices may use reference picture reordering and associate Ndifferent WP parameters to each of N regions. Then, for each block, thecorresponding reference index for motion estimation may be chosenaccording to its position. When a single block overlaps multipleregions, there are several criteria to decide which region the block isincluded in. In the current embodiment, we used top-left block positionfor the decision. Thus, in this case, the block is in the region wheretop-left pixel of the block is included.

The region-based WP can be extended to block based or macro-block (MB)based weighted prediction with marginal computation increase, butwithout any increase of memory bandwidth. In another embodiment,macro-block based weighted prediction are calculated with marginalcomputation increase, but without any increase of memory bandwidth.Thus, global brightness change detection is improved. Improving globalbrightness change detection (GBD) in order to produce a more accuratedecision. If there is no brightness change or global brightness change,on/off weighted prediction switch is switched globally. If we detectlocal brightness change, we apply weighted prediction with referencelist reordering to put the same reference in multiple reference list.With reference reordering, both region-based WP and MB-based weightedprediction are enabled.

As a result, the quality loss by improved GBD method is minimized. Theimproved GBD is also critical to region-based WP. By carefull decisionon the use of region-based WP, the coding gain is maximized byregion-based WP and the overhead bits to encoder reference index isminimized. The proposed MB-based WP with best MV cost check requiresmotion estimation only with 1 reference for each block, but have thesimilar performance with microblock based WP with motion estimation with2 references. Hence, better quality is achieved with marginal increaseof computational complexity.

MB-based WP with final MV cost check: If Fade=1 andN_(min)<Rcount<N_(max), we reorder the reference frame so that the samereference frames sits in two places, refidx=0 and refidx=1,respectively.

WP parameters are calculated and associated to refidx equal to 0 andassociate zero WP parameters to refidx equal to 1. The encoder shouldsend reference reordering syntaxes for decoders. When encoding each MB,motion estimation with zero WP parameters (refidx=1) are calculatedfirst and get the best MV and its MV cost, which are denoted by MV₁ andMVCost₁. Then, for MV₁, MV cost w.r.t. non-zero WP parameters (refidx=0)are calculated (denoted by MV₀ and MVCost₀). Decide the best referenceindex by comparing MVCost₁ and MVCost₀.

In the above algorithm, this is an issue on which reference index wewill use for motion estimation. Motion estimation gets better when theactual content is not hampered by brightness change. Based on this fact,we can do motion estimation with non-zero WP in case of fade-in (i.e.v_(cur)(n)−v_(ref)(n)>=0). The above MB-based WP can be modified to use2 references for motion estimation instead of final MV cost check. It issupposed to show the best performance. But the computational complexitywill increase. We use this 2-ref MB-based WP as bench mark to evaluatethe region-based WP and the (1-ref) MB-based WP with final MV costcheck.

While the foregoing is directed to embodiments of the present invention,other and further embodiments of the invention may be devised withoutdeparting from the basic scope thereof, and the scope thereof isdetermined by the claims that follow.

1. A method of a digital processor for determining a region-basedweighted prediction with improved global brightness detection, themethod comprising: applying a global brightness change detection methodsby computing the weighted prediction parameters; determining if thebrightness change is different amount of change for the differentregions; if the change is not different, calculate motion estimation andsetting Refidx to 1 and setting Refidx is set to 0 when there is change.determining the best motion vector, motion vector cost and the bestreference input. determining a region-based weighted prediction withimproved global brightness detection based on th2 motion vector data. 2.The method of claim 1, wherein the method utilizes two reference frameswithout global brightness change.
 3. The method of claim 1, wherein themethod considers at least one of the regions where brightness havechanged or consider all regions for simplicity.
 4. The method of claim1, wherein N reference indices using reference picture reordering andassociate N different weighted prediction parameters to each of Nregions.
 5. The method of claim 4, wherein for each MB the correspondingreference index for ME according to its position is considered.
 6. Themethod of 1, wherein a MB is decided to be in the region where top-leftpixel of the MB is considered.
 7. An apparatus for determining aregion-based weighted prediction with improved global brightnessdetection, the apparatus comprising: means for applying a globalbrightness change detection methods by computing the weighted predictionparameters; means for determining if the brightness change is differentamount of change for the different regions; if the change is notdifferent, means for calculate motion estimation and means for settingRefidx to 1 and setting Refidx is set to 0 when there is change. meansfor determining the best motion vector, motion vector cost and the bestreference input. means for determining a region-based weightedprediction with improved global brightness detection based on thr motionvector data.
 8. The apparatus of claim 7, wherein the method utilizestwo reference frames without global brightness change.
 9. The apparatusof claim 7, wherein the method considers at least one of the regionswhere brightness have changed or consider all regions for simplicity.10. The apparatus of claim 7, wherein N reference indices usingreference picture reordering and associate N different weightedprediction parameters to each of N regions.
 11. The apparatus of claim10, wherein for each MB the corresponding reference index for MEaccording to its position is considered.
 12. The apparatus of 7, whereina MB is decided to be in the region where top-left pixel of the MB isconsidered.
 13. A non-transitory computer readable medium with computerinstruction, when executed perform a method for determining aregion-based weighted prediction with improved global brightnessdetection, the method comprising: applying a global brightness changedetection methods by computing the weighted prediction parameters;determining if the brightness change is different amount of change forthe different regions; if the change is not different, calculate motionestimation and setting Refidx to 1 and setting Refidx is set to 0 whenthere is change. determining the best motion vector, motion vector costand the best reference input. determining a region-based weightedprediction with improved global brightness detection based on the motionvector data.
 14. The non-transitory computer readable medium of claim13, wherein the method utilizes two reference frames without globalbrightness change.
 15. The non-transitory computer readable medium ofclaim 13, wherein the method considers at least one of the regions wherebrightness have changed or consider all regions for simplicity.
 16. Thenon-transitory computer readable medium of claim 13, wherein N referenceindices using reference picture reordering and associate N differentweighted prediction parameters to each of N regions.
 17. Thenon-transitory computer readable medium of claim 14, wherein for each MBthe corresponding reference index for ME according to its position isconsidered.
 18. The non-transitory computer readable medium of 13,wherein a MB is decided to be in the region where top-left pixel of theMB is considered.